RoboCup@Home: Results in Benchmarking Domestic Service Robots

Benchmarking robotic technologies is of utmost importance for actual deployment of robotic applications in industrial and every-day environments, therefore many efforts have recently focused on this problem. Among the many different ways of benchmarking robotic systems, scientific competitions are recognized as one of the most effective ways of rapid development of scientific progress in a field. The ROBOCUP@HOME league targets the development and deployment of autonomous service and assistive robot technology, being essential for future personal domestic applications, and offers an important approach to benchmarking domestic and service robots. In this paper we present the new methodology for benchmarking DSR adopted in RoboCup@Home, that includes the definition of multiple benchmarks (tests) and of performance metrics based on the relationships between key abilities required to the robots and the tests. We also discuss the results of our benchmarking approach over the past years and provide an outlook on shortand mid-term goals of @HOME and of DSR in general.